Frankensteining LLMs

We demonstrate how with comparably little effort, often even few GPU resources, one can manage to still adapt LLMs, instead of purely being satisfied with the results of the big players. TNG is not a research lab, yet we have been successfully publishing our own models and are still working in directions how to adapt even huge LLMs with very reasonable efforts.

For Haystack, this is admittedly a long-shot, but I was recommended to submit. While we do not cover adaption of LLMs to improve search capabilities, our talk could motivate to go that direction.

Fabian Klemm

TNG Technology Consulting GmbH

Fabian Klemm completed his doctorate at the TU Munich in the field of discrete optimization and applied geometry. In 2020 and joining TNG as a software consultant. Besides a lot of DevOps experience, Fabian has a developed a greater interest in AI and particular LLM topics. Fabian contributes to the TNG Skainet team that operates the TNG internal AI server rack and since 2024 Fabian has been working in the internal TNG AI research team that is responsible for the successful TNG DeepSeek Chimera models.